METHOD AND APPARATUS OF RECORDING GRAPHIC DATA ON A 

RECORDING MEDIUM 



FOREIGN PRIORITY 
[0001] The present invention claims priority under 35 

5 U.S.C. 119 on Korean Application No. 10-2003-003783 filed 

January 20, 2 003; the contents of which are incorporated by 

reference in their entirety. 

BACKGROUND OF THE INVENTION 

Field of the Invention 
10 [0002] The present invention relates to method and 

apparatus of recording graphic data constituting sub-pictures in 

a main video on a high-density recording medium such as a Blu- 

ray Disk ROM (BD-ROM) disk. 

2 . Description of the Related Art 

15 Description of the Related Art 

[0003] The standardization of new high-density read 

only and rewritable optical disks capable of recording large 

amounts of high-quality video and audio data has been 

progressing rapidly and new optical disk related products are 

2 0 expected to be commercially available on the market in the near 

future. The Blu-ray Disc Rewritable (called "BD-RE' in general) 



is one example of these new optical disks. 

[0004] A digital TV signal is broadcasted and recorded 
in the form of TS (Transport Stream) on a high-density recording 
medium such as a BD-RE. The TS is composed of TPs (Transport 
5 Packets) and each TP, as shown in FIG. 1, includes a data 

segment divided from a PES (Packetized Elementary Stream) packet 
that carries video or audio data. The TS recoded as above is 
reproduced and transmitted as recorded to a digital TV set or 
its video/audio data contained in PES packets is decoded to an 

10 A/V signal by a disk recording/reproducing apparatus. 

[0005] On the other hand, the standardization for a 
high-density read-only optical disk such as a Blu-ray ROM disk 
(called x BD-ROM 9 ) is still under way. A BD-ROM need to have user 
interfacing data for reproduction control of video contents 

15 recorded thereon. 

[0006] By the way, user interfacing need graphic data 
such as buttons and button function descriptive images for 
user's easy operation. Furthermore, graphic data is also 
necessary to provide additional information about main video. 

2 0 Therefore, an effective method how graphic data and its display 
control information are recorded on a high-density recording 
medium such as a BD__R0M is strongly demanded. 

SUMMARY OF THE INVENTION 
[0007] It is an object of the present invention to 

25 provide method and apparatus of organizing graphic data to be 
displayed on a main video page and its display control 
information into a PES packet and recording them onto a 



recording medium. 

[0008] It is another object of the present invention 

to provide method and apparatus of organizing graphic data to be 
displayed on a single video page and its display control 
5 information into a single PES packet and recording them onto a 
recording medium. 

[0009] It is another object of the present invention 

to provide method and apparatus of organizing a PES packet to 
include common color control information on all graphic objects 

10 to be displayed on a single video page. 

[0010] A method of recording graphic data on a 
recording medium in accordance with the present invention is 
characterized in that it comprises the steps of: receiving 
graphic data including a plurality of graphic objects; and 

15 organizing the plurality of graphic objects and color control 

information into a PES (Packetized Elementary Stream) packet and 
recording the PES packet on the recording medium, wherein the 
color control information is used in common for screen display 
by the plurality of graphic objects. 

2 0 [0011] An apparatus of recording graphic data on a 

recording medium in accordance with the present invention is 
characterized in that it comprises: first means for receiving 
video and/or audio data, and graphic data including a plurality 
of graphic objects; second means for organizing the video and/or 

25 audio data into a series of PES packets and organizing the 

plurality of graphic objects and color control information into 
a PES packet, wherein the color control information is used in 
common for screen display by the plurality of graphic objects; 



and third means for multiplexing the series of video and/or 
audio PES packets and the PES packet including graphic objects 
in the unit of transport packet and then recording on a 
recording medium. 



5 BRIEF DESCRIPTION OF THE DRAWINGS 

[0012] The above features and other advantages of the 
present invention will be more clearly understood from the 
following detailed description taken in conjunction with the 
accompanying drawings, in which: 
10 [0013] FIG. 1 illustrates a recording example in which 

video or audio PES packets are recorded in series of transport 
packets after being sliced; 

[0014] FIG. 2 illustrates an exemplary embodiment of a 
recording medium file or data structure according to the present 
15 invention; 

[0015] FIG. 3 illustrates an example of a recording 

medium having the data structure of FIG. 2 stored thereon; 

[0016] FIG. 4 is diagram illustrating the structure of 
an optical disc apparatus where a method for 
20 reproducing/recording a multi-path data stream according to the 
present invention is applied; 

[0017] FIG. 5A illustrates a region including several 

graphic objects; 

[0018] FIG. 5B illustrates relations among graphic 

2 5 objects to be displayed on one video page, PES packets carrying 
them, and a series of transport packets to be recorded; 

[0019] FIG. 6 illustrates the structure of a PES 

4 



packet in accordance with a first embodiment of the present 
invention; 

[0020] FIG. 7 illustrates the structure of a PES 

packet in accordance with a second embodiment of the present 
5 invention; and 

[0021] FIG. 8 illustrates a graphic object that keeps 
to be displayed on a main video for an interval specified by the 
cut-in and cut-out time in accordance with the present 
invention . 

0 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0022] In order that the invention may be fully 

understood, preferred embodiments thereof will now be described 
with reference to the accompanying drawings. 

[0023] A high-density optical disk, for example, a 

5 Blu-Ray ROM (BD-ROM) , BD-RE, etc. in accordance with the 
invention may have a file or data structure for managing 
reproduction of video and audio data as shown in FIG. 2. Some 
aspects of the data structure according to the present invention 
shown in FIG. 2 are the same as the well-known BD-RE standard, 

0 as such these aspects will not be described in great detail. 

[0024] As shown in FIG. 2, the root directory contains 

at least one BD directory. The BD directory includes general 
files (not shown) , a PLAYLIST directory in which playlist files 
(e.g., *.mpls) are stored, a CLIPINF directory in which clip 

5 information files (*.clpi) are stored, and a STREAM directory in 
which MPEG2- formatted A/V stream clip files (*.m2ts), 
corresponding to the clip information files, are stored. 



[0025] The STREAM directory includes MPEG2- formatted 
single-path and/or multi-path A/V stream files that are called 
clips or clip files. The A/V stream includes source packets of 
video and audio. 
5 [0026] For example, a source packet of video data 

includes a header and a transport packet. A source packet 
includes a source packet number, which is generally a 
sequentially assigned number that serves as an address for 
accessing the source packet. Transport packets include a packet 

10 identifier (PID) . The PID identifies the sequence of transport 
packets to which a transport packet belongs. Each transport 
packet in the sequence will have the same PID. 

[0027] The CLIPINF directory includes a clip 
information file associated with each A/V stream file. The clip 

15 information file indicates, among other things, the type of A/V 
stream associated therewith, sequence information, program 
information and timing information. 

[0028] The PLAYLIST directory includes one or more 
playlist files. The concept of a playlist has been introduced to 

20 promote ease of editing/assembling clips for playback. A 

playlist file is a collection of playing intervals in the clips. 
Each playing interval is referred to as a playitem. The playlist 
file, among other things, identifies each playitem forming the 
playlist, and each playitem, among other things, is a pair of 

25 IN-point and OUT-point that point to positions on a time axis of 
the clip (e.g., presentation time stamps on an ATC or STC 
basis) . 

[0029] Expressed another way, the playlist file 

6 



identifies playitems, each playitem points to a clip or portion 
thereof and identifies the clip information file associated with 
the clip. The clip information file is used, among other things, 
to map the playitems to the clip of source packets. 
5 [0030] The general files (not shown) provide general 

information for managing the reproduction of the A/V streams 
recorded on the optical disk. 

[0031] In addition to illustrating the data structure 

of the recording medium according to an embodiment of the 

10 present invention, FIG. 2 represents the areas of the recording 
medium. For example, the general information files are recorded 
in one or more general information areas, the playlist directory 
is recorded in one or more playlist directory areas, each 
playlist in a playlist directory is recorded in one or more 

15 playlist areas of the recording medium, etc. 

[0032] FIG. 3 illustrates an example of a recording 
medium having the data structure of FIG. 2 stored thereon. As 
shown, the recording medium includes a file system information 
area, a data base area and an A/V stream area. 

2 0 [0033] The data base area includes a general 

information file and playlist information area and a clip 
information area. The general information file and playlist 
information area have the general information files recorded in 
a general information file area thereof, and the PLAYLIST 

25 directory and playlist files recorded in a playlist information 
area thereof. The clip information area has the CLIPINF 
directory and associated clip information files recorded 
therein. The A/V stream area has A/V streams and graphic data 



for the various titles recorded therein. 

[0034] Video and audio data are typically organized as 
individual titles; for example, different movies represented by 
the video and audio data are organized as different titles. 
5 Furthermore, a title may be organized into individual chapters 
in much the same way a book is often organized into chapters. 

[0035] FIG. 4 illustrates a schematic diagram of an 
embodiment of an optical disk recording and reproducing 
apparatus according to the present invention. As shown, an 

10 AV/Graphic encoder 9 receives and encodes data (e.g., movie 

video and audio data, bitmap- formatted graphic images, and their 
positions, colors and display effect control information, etc.) 
into PES packets . The AV/Graphic encoder 9 outputs the encoded 
data along with coding information and stream attribute 

15 information. A multiplexer 8 multiplexes the encoded data based 
on the coding information and stream attribute information to 
create, for example, an MPEG-2 transport stream. A source 
packetizer 7 packetizes the- transport packets from the 
multiplexer 8 into source packets in accordance with the 

20 audio/video/graphic format of the optical disk. 

[0036] The above-mentioned control information about 
positions, colors and display effects of graphic images may be 
entered through user's inputs to a controller 10 and then 
delivered to the AV/Graphic encoder 9 . 

25 [0037] As shown in FIG. 4, the operations of the 

AV/Graphic encoder 9, the multiplexer 8 and the source 
packetizer 7 are controlled by the controller 10. The controller 
10 receives user input on the recording operation, and provides 



control information to AV/Graphic encoder 9, multiplexer 8 and 
the source packetizer 7. For example, the controller 10 
instructs the AV/Graphic encoder 9 on the type of encoding to 
perform, instructs the multiplexer 8 on the transport stream to 

5 create, and instructs the source packetizer 7 on the source 
packet format. The controller 10 further controls a drive 3 to 
record on the optical disk the output from the source packetizer 
7 as a single clip file. 

[0038] The single clip file includes video and audio 

0 data and graphic-related data (graphic images, their positions, 
colors and display effect information) that are interleaved each 
other in the form of TS . 

[0039] The controller 10 also creates the navigation 
and management information for managing reproduction of the data 

5 being recorded on the optical disk. For example, based on 

information received via the user interface (e.g., instruction 
set saved on disk, provided over an intranet or internet by a 
computer system, etc.) the controller 10 controls the drive 3 to 
record one or more of the data structures of FIGS. 2 and 3 on 

0 the optical disk. 

[0040] During reproduction, the controller 10 controls 
the drive 3 to reproduce this data structure. Based on the 
information contained therein, as well as user input received 
over the user interface (e.g., control buttons on the recording 

5 and reproducing apparatus or a remote associated with the 

apparatus) , the controller 10 controls the drive 3 to reproduce 
the data from the optical disk. 

[0041] For example, as mentioned above with respect to 



the embodiments of the present invention, an A/V and graphic- 
related data stream included in a single clip file is reproduced 
based on the navigation information. 

[0042] The reproduced source packets of an A/V stream 
5 and a graphic-related data stream are received by a source 

depacketizer 4 and converted into appropriate data stream (e.g., 
an MPEG-2 transport packet stream) . A demultiplexer 5 
demultiplexes the respective data streams into PES packet- 
encoded data of video, audio and graphic. An AV/Graphic decoder 
10 6 decodes the encoded data encapsulated in PES packets to 

produce the original data that was fed to the AV/Graphic encoder 
9 or to produce video with which graphic images are sometimes 
overlapped. 

[0043] During reproduction, the controller 10 controls 
15 the operation of the source depacketizer 4, demultiplexer 5 and 
AV/Graphic decoder 6. The controller 10 receives user input on 
the reproducing operation, and provides control information to 
AV/Graphic decoder 6, demultiplexer 5 and the source 
depacketizer 4. For example, the controller 10 instructs the 
2 0 AV/Graphic decoder 9 on the type of decoding to perform, 
instructs the demultiplexer 5 on the transport stream to 
demultiplex, and instructs the source depacketizer 4 on the 
source packet format . 

[0044] While FIG. 4 has been described as a recording 
25 and reproducing apparatus, it will be understood that only a 

recording or only a reproducing apparatus may be provided using 
those portions of FIG. 4 providing the recording or reproducing 
function . 



[0045] The embodiments to organize the graphic-related 
data into a PES packet are described hereinafter. 

[0046] The organization of graphic-related data to a 
PES packet is made by main operation of the controller 10 in 
5 cooperation with the AV/Graphic encoder 9. However, the 

AV/Graphic encoder 9 can organize a PES packet by itself if 
necessary information for display control related with inputted 
graphic images are directly received. 

[0047] A single PES packet includes graphic-related 
10 data to be displayed on a single video frame (also called a 

'page' hereinafter) . One or more graphic regions are included in 
a single page and each graphic region includes at least one 
graphic image (called an ^object' hereinafter), e.g., PNG 
(Portable Network Graphic ) -formatted image, as shown in FIG. 5A. 
15 [0048] The time information of PTS (Presentation Time 

Stamp) and DTS (Decoding Time Stamp) for included objects is 
inserted in each PES packet corresponding to one page by the 
AV/Graphic encoder 9, as shown in FIG. 5B. 

[0049] FIG. 6 illustrates the structure of a PES 
20 packet organized by the controller 10 and the AV/Graphic encoder 
9 in accordance with a first embodiment of the present 
invention. As shown, the PES packet includes DTS (Decoding Time 
Stamp) for specifying the decoding time of graphic objects, PTS 
(Presentation Time Stamp) for specifying the presentation time 
2 5 of the graphic objects, and a plurality of color lookup tables 
(CLTs) that specify colors of graphic objects. Each CLT is 
composed of a palette member and a transparency array member 
that specify color and transparent level, respectively, of an 
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object or objects. Because a PES packet includes a plurality of -g 

1 

CLTs the graphic objects included in the PES packet can be s 
associated with different CLTs. 

[0050] The PES packet further includes region 
5 composition segments (RCSs) , object definition segments (ODSs) , 
PNG segments, and display control sequence segments (DCSSs). 

[0051] Each RCS includes the number of objects 
included therein, ID of each object, and information about the 
horizontal/vertical position at which each object is to be 
10 displayed. Each ODS includes an object ID, ID of an associated 
CLT, and PNG ID to identify a graphic image. Each PNG segment 
includes a PNG ID and a PNG chunk corresponding to an object 
image. Each DCSS includes display sequences of objects. 

[0052] Each display sequence includes the object ID, 
15 cut-in time and cut-out time information for display lasting 
time of the object, and multiple network graphic (MNG) chunk 
information . 

[0053] In the above embodiment of FIG. 6, different 
object IDs can be assigned to same shaped images that are 

2 0 displayed on same screen location. For example, a normal state 
button, a selected state one, and an action state one have same 
image altogether, however they need to be differently displayed 
in each state. Therefore, if three ODSs for the three-stated 
single button have different object IDs, different CLT IDs, and 

25 same PNG ID, respectively, the button can be displayed in 
different colors according to different states. 

[0054] FIG. 7 illustrates the structure of a PES 

packet organized by the controller 10 and the AV/Graphic encoder 

12 



9 in accordance with a second embodiment of the present 
invention. As shown, the PES packet includes DTS for specifying 
the decoding time of graphic objects, PTS for specifying the 
presentation time of the graphic objects, and a single global 
5 CLT that specifies color of all graphic objects. The global CLT 
is composed of a palette member and a transparency array member 
that specify color and transparent level, respectively, of all 
objects . 

[0055] The PES packet further includes region 

10 composition segments (RCSs) , object definition segments (ODSs) , 
PNG segments, and display control sequence segments (DCSSs). 

[0056] Each RCS includes the number of objects 
included therein, ID of each object, and information about the 
horizontal/vertical position at which each object is to be 

15 displayed. Each.ODS includes an object ID, and PNG ID to 

identify a graphic image. Because the global CLT is used in 
common, the ODS does not include CLT ID. Each PNG segment 
includes a PNG ID and a PNG chunk corresponding to an object 
image. Each DCSS includes display sequences of objects. 

20 [0057] Each display sequence includes an object ID, 

cut-in time and cut-out time information for display lasting 
time of the object, and multiple network graphic (MNG) partial 
palette chunk information. 

[0058] Because all objects use the same CLT in the 

25 second embodiment, the MNG partial palette chunk information is 
recorded to be used to change palette partially in order that an 
image object might be partially changed in color or display 
effect for outstanding display. 

13 



[0059] The cut-in time and cut-out time for display 
effects are defined in relative time with respect to the PTS 
written in a PES packet as shown in FIG. 8. 

[0060] As explained above, the cut-in time is time 
5 information to show subtitle such as a graphic object, and the 
cut-out time is time information to make a graphic object 
disappear. The time information is expressed in ticks. The ticks 
are ordinarily used in display of MNG- formatted images in 
synchronization of a main video frame. The expression in ticks 
10 provides better synchronization between a main video frame and 
its related graphic objects. 

[0061] For example, if a field of ticks per second is 
defined to 2997 and a video frame interval is defined to 100 
ticks, the frame rate becomes 29.97Hz (=2997 (ticks/sec) / 100 
15 (ticks/frame) ) . . 

[0062] In the example of FIG. 8, because the cut-in and 

cut-out time for a graphic object 801 including a text "Blu-ray" 
are respectively set to 100 and 3500 expressed in not physical 
time but the number of ticks, the graphic object 8 01 is 
20 displayed just a frame after its main video 800 whose PTS is set 
to ptsl is presented. Then, the object 801 keeps displayed state 
for 34 video frames and disappears at the 3 5-th video frame. 

[0063] The above- explained method and apparatus of 
recording graphic data on a high-density recording medium in 
2 5 accordance with the present invention ensures effective 

recording of graphic images e.g., icons for reproduction control 
and/or auxiliary information windows for video contents recorded 
on a recording medium. Furthermore, the present method is 

14 



capable of minimizing the size of graphic related data if color 
control information is used in common for a plurality of graphic 
obj ects . 

[0064] While the invention has been disclosed with 
5 respect to a limited number of embodiments, those skilled in the 
art, having the benefit of this disclosure, will appreciate 
numerous modifications and variations therefrom. It is intended 
that all such modifications and variations fall within the 
spirit and scope of the invention. 
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